Microscopy System and Method for Monitoring Microscope Activity

ABSTRACT

A microscopy system comprises a microscope for analyzing a sample, a computing device for processing measurement signals and at least one microphone for capturing sounds. The computing device is configured to evaluate captured sounds in order to identify a microscope activity in progress or command an intervention in the microscope activity in progress or identify ambient sounds based on microscope sounds.

REFERENCE TO RELATED APPLICATIONS

The current application claims the benefit of German Patent ApplicationNo. 10 2021 114 038.2, filed on 31 May 2021, which is herebyincorporated by reference.

FIELD OF THE DISCLOSURE

The present disclosure relates to a microscopy system and a method formonitoring microscope activity.

BACKGROUND OF THE DISCLOSURE

The automatic monitoring of microscope components is playing anincreasingly important role in modern microscopy systems. Microscopysystems are designed, for example, to be able to monitor a sample stageposition or the presence of a DIC slider (DIC: differential interferencecontrast) or of certain objectives at an objective revolver. Enhancedsafety is possible when a collision between, e.g., an objective of themicroscope and a sample or sample carrier can be prevented or at leastdetected early. If a plurality of steps are executed successively in aworkflow, especially automatically, it is desirable to be able tomonitor the implementation of the steps and, where necessary, perform acorrective intervention. An automatic monitoring can also increase userfriendliness, for example by automatically initiating a calibration modewhen a user places a calibration sample on a sample stage of themicroscope.

Monitoring systems of this type are described, inter alia, in DE 10 2017111 718 A1, DE 10 2017 109 698 A1, DE 10 2019 114 117 B3 and DE 10 2020210 595.2. Monitoring occurs here primarily by means of a camera, inparticular an overview camera, which is provided in addition to a samplecamera of the microscope. Any monitoring possible is limited to thefield of view of the camera, whereby it is often only possible tomonitor the area close to the sample. Adequate lighting conditions arealso required so that an active illumination is potentially necessary.

In addition to an overview camera, an electrical component monitoringcan also be implemented, for example by means of the “AutomaticComponent Recognition” technology (ACR) of the Applicant with which thecomponents to be monitored are electrically contactable in order totransmit, inter alia, identification signals. It is also possible toidentify appropriately designed microscope components by means of RFID.However, these approaches require specially designed components, whichinvolves higher costs and potential compatibility issues. The risk ofmalfunction is also potentially higher, for example when a circuit boardwith electronic components is required. In some cases, a retrofitting isalso costly or hardly feasible.

In principle, a monitoring can also occur by means of motion sensors,e.g., for detecting vibrations or collisions. However, this type ofmonitoring is limited to a relatively small number of monitorableproperties. There is thus an ongoing need to provide options formonitoring activity or components on a microscope.

SUMMARY OF THE DISCLOSURE

It can be considered an object of the invention to provide a microscopysystem and a method with which activity at a microscope can be monitoredparticularly precisely and with simple means in order to determine ameasurement situation with accuracy or facilitate, where appropriate, anecessary intervention.

This object is achieved by means of the microscopy system with thefeatures of claim 1, the microscopy system of claim 11 and by means ofthe method with the features of claim 21.

An embodiment of a microscopy system according to the inventioncomprises a microscope for analyzing a sample, a computing device forprocessing measurement signals and at least one microphone for capturingsounds. The computing device is configured to evaluate captured soundsin order to identify a microscope activity in progress based onmicroscope sounds or command an intervention in the microscope activityin progress.

In a method for monitoring microscope activity according to theinvention, a microscope is operated, sounds are captured by at least onemicrophone, and the captured sounds are evaluated in order to identify amicroscope activity in progress or command an intervention in themicroscope activity in progress based on microscope sounds.

Microscope sounds can in particular be sounds generated by or onmicroscope components, e.g., sounds of a microscope component inoperation, sounds caused by a movement of a microscope component, orsounds caused by a user on or with a microscope component. These kindsof microscope sounds are characteristic of an activity currently inprogress and thus permit an inference as to which microscope componentis being used, a property of the microscope component or a property ofthe activity currently in progress involving the microscope component.

A retrofitting of conventional microscopy systems is relatively easy tocarry out via the addition of at least one microphone and the adaptationof a computing device (in particular its software) to evaluate capturedsounds. In contrast to the electrical component recognition (ACR)mentioned above, it is possible to determine specific activities inprogress while compatibility problems are essentially circumvented.Compared to known camera-based monitoring systems, the acousticmonitoring of the invention can allow activities or states to bedetermined that are difficult or impossible to capture visually: forexample, sounds of a collision between microscope components can oftenbe determined when it is still not clear by means of a camera monitoringwhether there is still a small gap or already contact between twocomponents. An activity status or defect can potentially be inferredfrom sounds of a microscope component in operation, e.g. a scanner,while a camera can merely see the housing of the microscope component.

Using captured sounds (audio signals), a microscope activity that iscurrently in progress can be identified and/or an intervention in themicroscope activity in progress commanded. For example, a microscopeactivity “objective collision” can be identified from sounds of acollision between an objective and a cover slip and an interventioncommanded based on this identification, for example the intervention“stop component movement”. Alternatively, it is also possible for theevaluation of captured sounds, for example of the collision sounds, toresult directly in the commanding of an intervention, for example “stopcomponent movement”, without the microscope activity “objectivecollision” being explicitly labelled or output beforehand.

A further embodiment of a microscopy system according to the inventioncomprises a microscope for analyzing a sample, a computing device forprocessing measurement signals and at least one microphone for capturingsounds. The computing device is configured to evaluate captured soundsin order to identify ambient sounds.

Optional Embodiments

Variants of the microscopy systems according to the invention and of themethod according to the invention are the object of the dependent claimsand are explained in the following description.

Detection of a Defect, an Operating State or a Collision

The computing device can be configured to infer a defect of a microscopecomponent or that a microscope activity is not proceeding correctly,based on the microscope sounds. A microscope component can be, forexample, a sample stage, an objective revolver, a screw-on or otherwisereleasably attachable component, a laser scanner or an immersion device.Microscope sounds in these cases can be unusual sounds of a movingsample stage, unusual sounds during the changing of objectives, unusualfrequencies during a scan of the laser scanner or a scanning mirror, arattling of a screw-on or releasably attachable component or pumpingsounds of an immersion unit, which are an indication of bubbles or airin the pumped medium.

Additionally or alternatively, the computing device can be configured toinfer an operating state of the microscope based on the microscopesounds, in particular an operating state of a microscope component orinterchangeable component of the microscope. For example, the computingdevice can be designed to detect one or more of the following as anoperating state:

-   -   a differential interference contrast mode, based on the sound of        a DIC slider clicking into place as the microscope sound;    -   a mounting of an objective, based on the sound of an objective        being screwed into an objective revolver as the microscope        sound;    -   a sample stage movement, based on the sound of an operation of a        (in particular manually operated) sample stage as the microscope        sound;    -   a defective support of a microscope component, based on        microscope sounds characteristic of a loose support of the        microscope component. For example, worn connections can give        rise to a rattling, or an operation sound that occurs during        normal operation can be different;    -   an incorrect condenser position, based on movement sounds of a        swivel arm of a condenser when no sound of a complete pivoting        of the swivel arm into place is detected.

A condenser can be removed from the light path of the microscope bymeans of a swivel arm in order to, e.g., improve the accessibility ofthe sample area. This can be desirable in order to position a samplecarrier. The condenser must subsequently be swivelled back into thelight path by means of the swivel arm. If sounds of a swivel movementare captured but the sounds that typically occur in the event of acomplete swivel movement into the light path are not detected, then theswivel arm has most likely not been properly moved all the way back. Itis thus possible to detect such an incorrect position of the condenseracoustically.

Additionally or alternatively, the computing device can also beconfigured to detect one or more of the following as microscope soundsand corresponding identified microscope activities:

-   -   cleaning sounds of a slide cleaning activity; for example, a        wiping of a slide and/or breathing on a slide by a user, or        rinsing sounds of an automated cleaning activity in which, e.g.,        an immersion medium is rinsed off.    -   sounds of an application of an immersion medium; it is        optionally possible to distinguish based on these sounds between        a correct immersion activity and an incorrect immersion activity        in which air bubbles get into the immersion medium. An incorrect        immersion activity can be caused, for example, by a defective        tube or by an essentially empty tank and identified by means of        sounds that differ from a normal immersion activity.    -   insertion sounds of a sample carrier insertion activity on a        sample stage; it is optionally possible to distinguish based on        the insertion sounds at least between an insertion of a glass        slide, a plastic Petri dish and a microtiter plate, or in any        event between different materials of the sample carrier.    -   movement sounds of a filter wheel with filters being rotated in        or out of a microscope light path; for example, a light source        can include a filter wheel, wherein an adjustment of the filter        wheel can be established based on the characteristic rotation        sounds. Movements of other filters adjusted by a motor or        actuator can also be identified by the sounds that occur during        such an adjustment.

Additionally or alternatively, the computing device can also beconfigured to detect a component collision based on the microscopesounds, in particular between an objective and a sample, between anobjective and a sample carrier or cover slip of the sample carrier; orbetween an objective and a sample stage. A collision between anobjective and a sample stage can be identified, e.g., by characteristicscratching sounds. In collisions between an objective and a cover slip,there is initially a grating sound (crunch sound) followed by, if thepressure continues to increase, a breaking of the cover slip. Thecomputing device can be configured to detect a collision between anobjective and a cover slip early based on a grating as the microscopesound and to stop a component movement when such a collision is detectedin order to prevent the cover slip from breaking.

In order to improve the interpretation of captured sounds, it is alsopossible to evaluate combinations of simultaneous or successivemicroscope sounds deriving from different microscope components oractivities. For example, a collision is more likely if it is accompaniedor preceded by a sample stage movement. If a microscope sound of acollision overlaps a microscope sound of a sample stage movement (e.g.characteristic sounds of a motor or friction) and/or if sounds of asample stage movement precede the collision sound, then the computingdevice can infer a collision with a higher probability.

Ambient Sounds

The computing device can optionally be configured to take into accountor identify both microscope sounds and ambient sounds from capturedsounds. The variants described in the following are applicable toembodiments of the invention in which the computing device is onlyconfigured to be able to identify ambient sounds (but not microscopesounds) as well as embodiments in which the computing device isconfigured to identify both ambient sounds and microscope sounds.

In particular, ambient sounds can be sounds which are not produced on orby a component of the microscopy system and/or which are not produced byan operation of the microscopy system. Rather, ambient sounds can inparticular be sounds the origin of which can be found in ambientinfluences or other objects which do not form part of the microscopysystem. Sounds of a human origin which are not related to the operationor control of the microscopy system can also be considered ambientsounds. Voice commands of a person intended to control the microscopeare thus optionally not considered ambient sounds. Ambient sounds can inparticular be ambient sounds characteristic of a potentially disruptiveexternal influence, e.g. resulting from vibrations of a floor,oscillations or changes in air pressure, or changes in an ambienttemperature, brightness or humidity. Ambient sounds can thus relate to,e.g., one or more of the following: a draft (air movement), a closing orslamming of a room door or window, footsteps, a kicking or stumbling ofa person, construction site noise, drilling sounds or a shock or blow tothe microscope system or to an object not forming part of the microscopesystem, which could potentially cause a vibration of the microscopesystem. Ambient sounds can also be the sounds produced by a pneumaticadjustment of a table on which the microscope is supported. For the mostpart, these are characteristic hissing sounds. A pneumatic adjustmentfrequently takes the form of an automatic regulation by means of asensor, which is intended to compensate any vibrations. Thecorresponding pneumatic sounds can thus indicate that a vibration hasoccurred. A further example of ambient sounds are air-conditioningsounds such as fan sounds, a humming or trickling sounds duringoperation, or a beep tone with which the air conditioner confirmsreceipt of a remote-control signal. Ambient sounds can also be caused bymanual operating activity on ambient devices, for example by theactivation of a light switch, a measurement device not forming part ofthe microscopy system or some other electronic device in the vicinity ofthe microscopy system. In particular, photography sounds of a camera canconstitute ambient sounds. This camera is not part of the microscopysystem, but can be, for example, a smartphone camera, wherein the soundsproduced are, e.g., an imitation of a camera shutter sound output by theloudspeaker of the smartphone. Further ambient sounds can have theirorigin in equipment or devices used during the operation of themicroscopy system. For example, the sounds of an opening or closing ofan incubator panel or door can be captured. The computing device canthereby monitor whether a user has closed the incubator panel correctlybefore any measurements are performed. An opening or closing of otherhousing doors can also be captured in order to infer a state of a deviceor component. Fan sounds can also be identified as ambient sounds,wherein the computing device can optionally distinguish betweendifferent devices comprising a fan/ventilator based on different fansounds.

The identification of an ambient sound can comprise a differentiationbetween different possible ambient sounds so as to specify a type ofambient sound or an ambient sound source. The computing device can thusidentify in particular an ambient activity, an ambient state or anexternal influence.

The computing device can be configured to log an identified ambientsound. A time and/or a reference to captured microscope measurement datacan be saved. For example, it can be recorded for captured image datawhether an ambient sound was detected at the time of capture of theimage. The ambient sound can be linked to the image data in the form ofan audio file; alternatively, it is possible to save solely anindication of the type of ambient sound with the image data that wascaptured at the moment of the ambient sound. This information can alsobe saved as metadata of an image or video file. For example, it can belogged for a microscope image that a closing of a door was determinedduring the capture of this image. This facilitates troubleshooting incases of deficient images. If it is recorded for captured microscopemeasurement data that a camera actuation sound was registered at thetime of the measurement, a user can correctly match the time of captureof the measurement data with the time of capture of an image generatedwith means which do not form part of the microscope system, for examplea mobile phone, which produced the camera actuation sound. It isconsequently possible for, e.g., a measurement configuration to belogged by means of the image from the mobile phone, this image beingcorrectly matched with the microscope measurement data captured with thephotographed measurement configuration.

The computing device can also be configured to present the user in theevent of certain identified ambient sounds with an option of repeating ameasurement performed concurrently with the ambient sound. Instead ofthe display of such an option, it is also possible for a repetition ofthe measurement to occur automatically. If, for example, the ambientsound takes the form of a loud blow likely to cause a disruptivevibration, an automatic repetition of the measurement can occur.

The computing device can also be configured to identify certain soundsas control signals of the microscope, for example as start and/or stopsignals for a measurement procedure or image capture. In principle, anysounds can be exploited for this purpose; for example, the sounds can beproduced by a person with an auxiliary device or by clapping. Thesesounds can optionally be considered ambient sounds or be identifiable bythe computing device in addition to the ambient sounds described above.

Situation-Dependent Acoustic Monitoring

In principle, the described acoustic monitoring can be continuous.Alternatively, the computing device can also be configured to only carryout a detection of microscope sounds and/or ambient sounds or anoperation of the microphone in response to a situation-dependentactivation signal, i.e. a continuous monitoring does not occur. Thecomputing device can generate the activation signal, e.g., for certainworkflows of the microscope and/or in certain states deduced from avisual monitoring. For example, if an overview camera establishes aproximity between an objective and a sample or sample carrier, anactivation signal can be generated in order to provide an additionalacoustic collision monitoring.

Contextual Information

The computing device can optionally be configured to exploit additionalcontextual information in the detection of microscope sounds or theidentification of a microscope activity in progress. The contextualinformation can stem, e.g., from an analysis of captured overview imagesor an analysis of measurement data of a (motion) sensor and/or beinformation regarding an initiated workflow of the microscope, amicroscope configuration used, an employed sample carrier or a currentmicroscope user. By identifying a microscope user, it is possible toanalyze, e.g., sounds of the cleaning of a cover slip as a function of auser in order to take different habits into account. Knowledge of amicroscope user can also provide clues as to a type of experimentconducted or which microscope components are frequently or never used bythe user. For example, different models of automatic immersion units candiffer in the sounds they produce; knowing which model the user owns canthus be advantageous in the interpretation of captured sounds.

The results of the described audio-based control/supervision can also becombined with results of other control methods using, e.g., an overviewcamera, a motion sensor or a contacting of electrical devices. Thecomputing device can evaluate results of these control methods togetherin order to determine, e.g., a microscope activity in progress or acurrent microscope state with a particularly high degree of reliability.Alternatively or additionally, a result of one control method can beused as contextual information in one of the other control methodsmentioned.

One or More Microphones and their Arrangement

Generally, one or more microphones can be provided. A microphone can bearranged directly on the microscope, e.g., on the stand, sample stage orobjective revolver, or can be located at a distance from the microscope.A location of the at least one microphone can be fixed relative to thestand or microscope, which facilitates an interpretation of capturedsounds. However, this is not mandatory and a variable position relativeto the microscope is possible, e.g., if the microphone or one of themicrophones is part of some other device, e.g., a smartphone, a screenor a monitoring camera independent of the microscope.

By using a plurality of microphones, the computing device is better ableto filter out background noise or interfering sounds stemming inparticular from other (microscope) systems located in the same room orhall. By evaluating captured sounds of the plurality of microphones, thecomputing device is also able to identify the microscope activity and/oran activity causing ambient sounds (ambient activity) depending on alocation of a sound source. The sound source can be, e.g., a microscopecomponent. This kind of location-dependent interpretation facilitates,for example, a determination of whether objective detachment/attachmentsounds or scan mirror sounds in fact stem from the microscope to whichthe microphones belong or from another microscope in the same room. Incases where an incubator panel is the sound source, the plurality ofmicrophones allow a more reliable distinction vis-à-vis sounds of otherpanels or panel activity involving a snap-in closure. An explicitlocalization of a sound source can take place, but is not mandatory. Forexample, a location-dependent identification of a microscope activitywithout an express localization of the sound source can be implementedin a machine-learned model by means of training data comprising soundswhich were generated at different locations. Similar sounds of thetraining data can be annotated differently for different locations. Amodel is thereby learned which is able to discriminate and classifysimilar sounds deriving from different locations, without knowing oroutputting the location of the sounds.

Active Sonar Method

The microscopy system can also comprise at least one sound transmitterin order to carry out an active sonar method. In principle, a soundtransmitter can be any sound-producing component, for example aloudspeaker, a vibratable membrane or some other vibrationally excitableobject. The computing device can be configured to control the soundtransmitter or sound transmitters to emit acoustic pulses, wherein theat least one microphone measures reflected acoustic pulses. Byevaluating the measured acoustic pulses, the computing device canestablish a presence or a location of objects and/or identify an object,for example as a function of its dimensions and local coordinatesdetermined by sonar.

Learned Model for Sound Evaluation

The computing device can be configured to evaluate captured sounds bymeans of at least one machine-learned model learned using training dataof sounds or microscope sounds. The training data accordingly comprisessounds which at least partially contain microscope sounds and/or ambientsounds. Different types of sounds can overlap in the training data.

The model can be learned by means of a supervised learning process inwhich the training data comprises different microscope sounds and/orambient sounds, which are respectively annotated with an indication ofan associated microscope activity or ambient property. Annotations arethus labels by means of which training data is divided into differentcategories, e.g., collision, snap-in closure, scanning motion, stagemotion or immersion activity. More than one label can be assigned to thesame training example, e.g., stage motion with collision. Other soundsof the training data which are not microscope sounds are accordinglyannotated differently, e.g., as ambient sounds or as sounds to beignored.

Training data at least partially containing microscope sounds cancomprise one or more of the following, in particular with acorresponding annotation:

-   -   cleaning sounds of a slide cleaning activity and optionally        other cleaning sounds that do not belong to a slide cleaning        activity;    -   sounds of an immersion device in the event of a correct        application of an immersion medium;    -   sounds of an immersion device in the event of an incorrect        application of an immersion medium;    -   insertion sounds of a sample carrier insertion activity on a        sample stage and optionally other sounds produced with the        sample carrier when placed on a substrate without an insertion        activity being performed;    -   sounds of a shock or blow to the microscopy system and        optionally sounds of a shock or blow which does not directly        affect the microscopy system;    -   collision sounds of microscope components, in particular        including a grating of a collision between an objective and a        cover slip, breaking sounds of a cover slip in the event of a        collision with an objective, sounds of a collision between an        objective or a condenser and different types of sample carriers,        sounds of a collision between an objective and a sample stage;    -   sounds of a DIC slider snapping into a DIC slot/chamber on the        microscope; sounds of different filters snapping into        corresponding filter slots/chambers on the microscope; other        snap-in sounds unrelated to the microscope;    -   sounds of an objective being screwed into an objective revolver        and optionally other sounds produced by a threaded attachment        unrelated to the microscope;    -   movement sounds of a microscope component which is movable by a        motor or actuator, in particular movement sounds of a manual or        motorized sample stage in operation;    -   a rattling of an inadequately supported microscope component;    -   operating sounds of a microscope component in the event of a        correct operation and in the event of an incorrect operation,        wherein the microscope component is in particular a sample        stage, an objective revolver, an objective, an immersion device,        a laser scanner, a screw-on or otherwise releasably attachable        component.

As the training data also includes sounds similar to the sounds to bedetected but which do not derive from a microscope activity, it ispossible to learn to distinguish between these sounds with a high degreeof reliability, for example between a breaking of a cover slip and abreaking of some other object or glass.

The annotations labelling a microscope sound can also be linked to acommanding of an action or intervention or be replaced by suchannotations. For example, the grating sound of a collision between anobjective and a cover slip can be linked to the annotation “objectivecollision”, which labels a microscope activity, or to an annotationdesignating an action, e.g., “stop sample stage movement/componentmovement”.

Training data at least partially containing ambient sounds can includeone or more of the following, in particular with a correspondingannotation: draft sounds; construction site noise; drilling sounds;air-conditioning sounds; sounds of a door closing or slamming next tothe microscopy system; sounds of footsteps or of a person stumbling;hissing sounds of a pneumatic adjustment of a table on which the/amicroscope is supported; manual operating activity on ambient devices;photography sounds of a camera.

Sounds of the training data can be captured with a microscope ormicroscopy system as described in the present disclosure. For thispurpose, a plurality of microphones can be used simultaneously, asdescribed in the foregoing. Optionally, a pre-processing of capturedsounds can occur, wherein the training data and the sounds capturedduring operation which are to be evaluated are pre-processed in anidentical manner.

Instead of a single learned model, it is also possible for a pluralityof separate learned models to perform the tasks described here. Forexample, the aforementioned classification tasks can be distributedamong different models.

The learned model or a further learned model can also be learned bymeans of an unsupervised learning process. In an unsupervised learningprocess, the training data can comprise, e.g., different microscopesounds and/or ambient sounds captured during an error-free operation ofthe microscope. A model which detects sounds of a predetermined class ofnormal or expected sounds is learned thereby. In the event of soundsthat deviate from the training data, the model can identify a microscopeactivity in progress as an incorrect or potentially incorrect microscopeactivity. During a normal operation of the microscope without errors,sounds can be continuously captured and added to the training data inorder to better identify expected normal sounds.

Sounds captured during normal operation can also be added to thetraining data in a supervised learning, wherein an annotation can occurautomatically. This lends itself, for example, to predeterminedworkflows in which a user performs an activity and then confirms thesuccessful completion of the activity, for example, by means of an inputon the microscopy system or a computer. For example, a request can beissued in a workflow by the computing device to a user to position acalibration sample. The insertion sounds during the positioning of thecalibration sample are captured and, by means of the subsequentconfirmation of the performance of this activity by the user, theannotation can be generated that the captured sounds are sounds of aninsertion/positioning of a calibration sample.

A learned model can comprise one or more neural networks, e.g., an RNN(recurrent neural network), an LSTM network (LSTM: long short-termmemory) or an in particular one-dimensional or two-dimensional CNN(convolutional neural network). Captured audio signals can also beconverted into two-dimensional representations such as, e.g.,spectrograms or other images, which allows the use of modelarchitectures known from image processing. Other deep neural networkmodel architectures are also possible. A partially supervised trainingor a reinforcement learning is also possible.

In a supervised training, model parameters of the model are defined bymeans of a learning algorithm using the annotated training data. Apredetermined objective function can be optimized to this end, e.g., aloss function can be minimized. The loss function describes deviationsbetween the predetermined labels or annotations and current outputs ofthe model, which are calculated with the current model parameter valuesfrom entered training data. By means of this iterative minimization, themodel is able to generate outputs that are closer and closer to thepredetermined labels. The model parameter values are modified tominimize the loss function, which can be calculated, e.g., by gradientdescent and backpropagation. In cases of a CNN, the model parameters canin particular comprise entries of convolution matrices of the differentlayers of the CNN.

Monitoring of Workflows and Actions

Results of the described audio control can be exploited for themonitoring of activity and/or the derivation of actions to beimplemented/interventions in an activity in progress.

In particular, the computing device can be configured to monitorworkflows, in which capacity it checks whether measured sounds aretypical of a predetermined workflow. The computing device can alsomonitor manual steps of a workflow by checking whether sounds caused bya user on the microscope are typical of a current step of apredetermined workflow and optionally when the step is completed. If itis established based on the acoustic control that a microscope activityto be performed manually (e.g. following a request to the user tochange, insert or attach something on the microscope) has beencompleted, the workflow can be continued automatically without a requestfor confirmation being issued to the user.

An action in the event of an identification of a certain microscopeactivity can also take the form of an outputting of warnings or requestsfor information to a user, a pausing of a workflow or an immediatestoppage of a current activity, e.g., a sample stage movement.

A verification of the audio-based identification of a microscopeactivity through some other monitoring modality can also be provided asan action. For example, in the event of an identification of amicroscope activity based on the microscope sounds, it is possible tocommand an evaluation of an overview image in order to verify or refinethe identification of the microscope activity. The overview image canalready have been captured beforehand or be captured in response to theaudio-based identification. For example, if captured sounds suggest thereplacement of a sample carrier, a new overview image can be capturedand evaluated with respect to the location and/or type of a presentsample carrier.

If the computing device establishes a (in particular manual) movement ormanipulation of a sample, sample carrier or sample stage of themicroscope from the measured sounds, the computing device canaccordingly optionally command a new calibration or a calibrationcontrol, e.g. by evaluating an overview image.

Moreover, the identification of an acoustically established microscopeactivity as well as related information can be recorded in a log file,e.g. for a subsequent error analysis, or transmitted to a microscopemanufacturer, for example, in order to inform a technical service aboutan issue that has occurred.

General Features

The measurement signals processed by the computing device comprise thesounds captured by the microscope and can optionally further compriseany other measurement signals of the microscope, e.g., overview imagesor sample images.

A microphone can be of essentially any technical design and can captureinfrasound or ultrasound in addition or alternatively to audible sound.Besides the narrower definition of a microphone as an airborne soundtransducer designed to measure air pressure changes, a microphone canalternatively or additionally also be understood as an electroacoustictransducer that measures mechanical vibrations in solids. Such atransducer can be used to capture vibrations of a body coupled to themicrophone, such as a stand, sample stage, objective revolver or someother microscope component on which the transducer is mounted. It is inparticular possible to use a piezoelectric acoustic pickup.

Microscope sounds can be understood as the sounds caused by or on amicroscope component. These include, e.g., sounds produced in the eventof contact between the microscope component and another component oroperating sounds of the microscope component. They can also be soundscaused by a user acting on the microscope component, e.g. when moving orcleaning a sample carrier or when changing microscope configurations.Voice commands with which a user can control an electronic device in anessentially known manner are accordingly not microscope sounds.

In the evaluation of captured sounds, it is possible to first identify amicroscope sound and then deduce a microscope activity from theidentified sound. Alternatively, there is no explicit labelling of themicroscope sounds but rather a determination of a microscope activitydirectly from the captured sounds. This can be the case, e.g., withmachine-learned models when an input into the model is the capturedsound and an output of the model is an identified microscope activity.In particular with learned models, the identification of a microscopeactivity currently in progress and the instruction of an action can alsobe combined in a single output. For example, if a sound of a collisionbetween an objective and a sample carrier is captured, a trained modelcan emit an output “stop collision through stoppage of current componentmovement” directly. In still further variants, an action, e.g. “stopcurrent component movement”, is commanded directly with the evaluationof captured sounds without an explicit identification or labelling ofthe captured microscope sounds or microscope activity associated withthe same. For the purposes of illustration, different examples of theinvention which involve an identification of a microscope activity inprogress are described. The examples can, however, also be varied suchthat, instead of an identification of a microscope activity in progress,an action or an instruction for an intervention in the microscopeactivity occurs directly.

A microscopy system denotes an apparatus comprising at least onecomputing device and a microscope. A microscope can in particular beunderstood as a light microscope, an X-ray microscope, an electronmicroscope, an atomic force microscope or a macroscope. The operation ofthe microscope can in particular comprise a measurement operation, acalibration operation or preparatory measures for carrying out asubsequent measurement operation. Preparatory measures can comprise,e.g., a cleaning, positioning, setting or connecting of (microscope)components.

The computing device can be physically part of the microscope orarranged separately in the vicinity of the microscope or at a locationat any distance from the microscope. The computing device can also bedesigned to be decentralized. It can generally be formed by anycombination of electronics and software and can comprise in particular acomputer, a server, a cloud-based computing system or one or moremicroprocessors or graphics processors. The computing device can also beconfigured to control a sample camera, an overview camera, an imagecapture, a sample stage movement and/or other microscope components.

An analyzed sample can of any kind including, for example, biologicalcells or cell parts, material samples or rock samples, electroniccomponents and/or objects held in liquid.

A computer program according to the invention comprises commands that,when executed by a computer, cause the execution of one of the describedmethod variants.

The characteristics of the invention that have been described asadditional apparatus features also yield, when implemented as intended,variants of the method according to the invention. Conversely, amicroscopy system can also be configured to carry out the describedmethod variants. In particular, the computing device can be configuredto carry out the described method variants and/or output commands forthe execution of described method steps. The computing device can alsocomprise the described computer program. While some variants use aready-trained model, other variants of the invention result from theexecution of the corresponding training steps.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the invention and various other features andadvantages of the present invention will become readily apparent by thefollowing description in connection with the schematic drawings, whichare shown by way of example only, and not limitation, wherein likereference numerals may refer to alike or substantially alike components:

FIG. 1 shows schematically an example embodiment of a microscopy systemof the invention;

FIG. 2 illustrates schematically an example embodiment of a method ofthe invention;

FIG. 3 illustrates schematically a training of a model of the method ofFIG. 2 ;

FIG. 4 illustrates schematically a further example embodiment of amethod of the invention;

FIG. 5 shows schematically a further example embodiment of a microscopysystem of the invention;

FIG. 6 illustrates schematically a further variant embodiment of amethod according to the invention, which is carried out by an exampleembodiment of a microscopy system of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

Different example embodiments are described in the following withreference to the figures.

FIG. 1

FIG. 1 shows an example embodiment of a microscopy system 100 accordingto the invention. The microscopy system 100 comprises a computing device70 and a microscope 1, which is a light microscope in the illustratedexample, but which in principle can be any type of microscope. Themicroscope 1 comprises a stand 2 via which further microscope componentsare supported. The latter can in particular include: an illuminationdevice 16; an objective changer/revolver 3, on which an objective 4 ismounted in the illustrated example; a sample stage 6 with a holdingframe for holding a sample carrier 7 and a microscope camera 8. When theobjective 4 is rotated into the light path of the microscope, themicroscope camera 8 receives detection light from a sample area in whicha sample can be located in order to capture a microscope image. Samplescan be any object, fluid or structure. The microscope 1 also comprisesan optional overview camera 9 for capturing an overview image of asample environment. A field of view 9A of the overview camera 9 islarger than a field of view when a sample image is captured. In theillustrated example, the overview camera 9 views the sample carrier 7via a mirror 9B. The mirror 9B is arranged on the objective revolver 3and can be selected instead of the objective 4. In variations of thisembodiment, the mirror can also be arranged at a different location, orthe overview camera 9 is arranged so as to view the sample carrier 7directly without a mirror 9B.

It is desirable to monitor microscope activities in progress in order toimprove automatic processes, increase user friendliness, detect errorsearly and enable a correction of such errors where necessary. This canoccur in part by means of the optional overview camera 9. In contrast tomonitoring methods of the cited prior art, at least one microphone 5 isemployed for monitoring. In the illustrated example, a microphone array5A of a plurality of microphones 5 is used, which facilitates adirection-dependent and/or distance-dependent evaluation of capturedsounds. In the illustrated case, the microphones 5 are arranged on thestand 2, although a positioning on other microscope components or at adistance from the microscope 1 is additionally or alternatively alsopossible.

Sounds captured by the at least one microphone 5 are evaluated by thecomputing device 70. The computing device 70 comprises a correspondingcomputer program 80 for this purpose. The computing device 70 orcomputer program 80 is configured to execute one of the examples of amethod according to the invention described in the following.

FIG. 2

FIG. 2 shows schematically a process sequence of an example embodimentof a method of the invention. A microscope sound 25 is produced by amicroscope activity in progress/process P1. In the illustrated example,the microscope activity P1 is a collision between the objective 4 andthe sample carrier 7 due to a height adjustment 20 of the sample stage6. The collision causes sound waves, i.e. microscope sounds 25,characteristic of this microscope activity P1.

In process P2, the at least one microphone captures audio signals/sounds22 containing the microscope sounds 25. In the sounds 22, the microscopesounds 25 can overlap with, e.g., ambient sounds or other microscopesounds, for example operating sounds of a sample stage motor.

In process P3, the captured sounds 22 are evaluated, optionally after apre-processing. In the illustrated example, the sounds 22 are input intoa machine-learned model M to this end. The model M is trained toclassify input sounds 22. An output 30 generated in process P4 by themodel M is thus an identification 31 or indication of a classificationof a microscope activity in progress that caused the microscope sounds25. In this example, the output 30 or identification 31 thus indicates acollision between an objective 4 and a sample carrier 7.

In process P5, an action (an intervention) 34 occurs based on theidentified microscope activity, i.e. based on the output 30 of the modelM. The intervention 34 can be a control of a microscope component. Inthe illustrated example, a stop command 35 is output in order to preventa further movement of the sample stage 6. A breaking of the samplecarrier 7 or cover slip of the sample carrier 7 can thereby potentiallybe avoided when microscope sounds 25 in the form of grating sounds of aninitial contact are evaluated early.

The model M implemented here is described in more detail with referenceto FIG. 3 .

FIG. 3

FIG. 3 shows schematically the model M of FIG. 2 together with trainingdata T1-T5 by means of which model parameters of the model M are definedin a supervised learning process.

The model M can be, e.g., an RNN (recurrent neural network). Soundscaptured, e.g., by the at least one microphone 5 are used as trainingdata T1-T5. An annotation A is specified for each sound, the annotationA indicating whether or what kind of microscope sound 25 is contained inthe respective training data. In the illustrated example, the trainingdata T1-T4 respectively contain microscope sounds 25 deriving from acollision between an objective and a sample carrier, which is indicatedaccordingly in the annotations A. The training data T5, on the otherhand, comprises other sounds produced, for example, by a sample carrierbeing pressed against other objects. This is likewise recorded by meansof the annotations A. In the learning process, a predetermined learningalgorithm iteratively adjusts parameters of the model M so that theoutputs generated by the model M for the training data T1-T5 match thepredetermined annotations A better and better. By means of the soundsand annotations A used here by way of example, the model M thus learnsto distinguish between collision sounds which can occur between anobjective and a sample carrier during operation of the microscope andother sounds which occur when other objects collide.

The described collision detection with subsequent stoppage of acomponent movement is merely a concrete example of a method according tothe invention. The control device 70 can also be designed to be able todetect other or further microscope sounds 25. Microscope sounds 25 canbe, for example, the sounds produced when a microscope component ismoved, for example, a (manual) sample stage movement, a movement of aswivel arm or a turning of the objective revolver 3, or sounds producedby a microscope component in operation, for example operating sounds ofa scanner or immersion device, or sounds produced on a microscopecomponent by a user, for example when screwing an objective 4 into anobjective changer 3 or when positioning a sample carrier 7 on the samplestage 6.

With reference to FIG. 2 , the stop command 35 in process P4 is merelyan example of an intervention 34 carried out based on the output 30 ofthe model M. Other interventions/actions 34 can relate to a control ofbasically any microscope component or to instructions to a user to carryout certain activities. In particular, a completion of individual stepsto be performed by a user in a multi-step workflow can be monitored: forexample, a user can be invited to set up a new sample carrier 7. Ifcorresponding microscope sounds 25 (e.g. sounds of an insertion of asample carrier 7 on a holding frame of the sample stage 6) are detected,it is possible to proceed automatically to the next step in the workflowwithout the user having to issue a command to proceed to the next step.A further example of a method according to the invention is describedwith reference to the following figure.

FIG. 4

FIG. 4 schematically shows a process sequence of a further exampleembodiment of the method according to the invention. In this example,the microscope 1 is equipped with an immersion device 11 comprising,inter alia, an immersion medium tank 12, a pump (not illustrated) and animmersion medium line 13, for example a gooseneck, a tube and/or a rigidcannula. An immersion medium is conveyed from the immersion medium tank12 to a front side of the objective through the immersion medium line13. Characteristic microscope sounds 25 are produced by the operation ofthe pump and by the movement of the immersion medium through and out ofthe immersion medium line 13. In the event of a malfunction (microscopeactivity P1′), for example if the immersion medium line 13 is blocked orthe pump draws air from the immersion medium tank 12, the resultingmicroscope sounds 25 are different.

The at least one microphone again captures sounds 22′ containing themicroscope sounds 25′ of the immersion device 11 caused by themicroscope activity P1′ (process P2′).

In this example, the captured sounds 22′ are processed first beforebeing evaluated by a learned model M′. The sounds 22′ can be reproducedin a representation in which an amplitude is saved for successive pointsin time as, e.g., in standard pulse-code modulation (PCM) methods. Thisrepresentation can be converted in process P2 b into an audiospectrogram 26 representing a progression of different frequencies overtime. The level of each frequency can be, e.g., colour-modulated or berepresented by grey values or a line thickness. Such a spectrogram 26 orprocessed sounds 27 in general can be represented as a two-dimensionalimage. This image is input into a learned model M′ designed, forexample, as a CNN.

The learned model M′ processes the input spectrogram 26 (process P3′)and generates an output 30 (process P4′) therefrom. The output 30 inthis case is, however, not an identification of the microscope activitythat caused the microscope sounds 25 (i.e. not an identification:“malfunction of the immersion medium device 11”). Instead, the model M′outputs an instruction for an action 34 intended for this microscopeactivity directly. In this example, a user is invited to check a filllevel of the immersion medium tank 12 and/or to replace the immersionmedium line 13 as the action/intervention 34 (reference sign 36).

Training data of the model M′ can also be expediently represented in theform of spectrograms. The training data in this case can comprisespectrograms of microscope sounds produced during a correct operation ofthe immersion device 11. The training data can additionally containspectrograms which correspond to microscope sounds captured during anincorrect operation of the immersion device 11. The training data can beannotated accordingly for a supervised learning process. Alternatively,in an unsupervised learning process, the training data can comprisesolely spectrograms relating to a correct operation of the immersiondevice 11. The model can thereby be trained to detect anomalies, i.e.sounds that differ from the sounds produced in the event of a correctoperation of the immersion device 11.

The described examples of the figures can also be combined. For example,the example of FIG. 2 can be varied such that the learned model M alsoprocesses spectrograms and/or such that the learned model M outputs thestop command 35 or, more generally, an instruction for an intervention34 in the microscope activity directly instead of an identification 31of a microscope activity currently in progress.

FIG. 5

FIG. 5 shows schematically a further example embodiment of a microscopysystem 100 according to the invention. Compared to the example shown inFIG. 1 , the microscopy system 100 here additionally comprises a soundtransmitter 15 in order to facilitate an active sonar method. In theillustrated example, the sound transmitter 15 is arranged on the stand2. In principle, the sound transmitter 15 or further sound transmitterscan also be provided at a different location or in a differentorientation.

The computing device 70 controls the sound transmitter 15 to emitacoustic pulses 17. Any acoustic pulses 17 reflected by objects can thenbe measured by the at least one microphone 5. An evaluation of thesemeasured acoustic signals by the computing device 70 allows an inferenceas to the location, size and/or shape of a sound-reflecting object. Theobject can be a microscope component or objects arranged in the samplearea such as, e.g., a sample or the sample carrier 7.

Compared with the overview camera 9, the sonar method provides another,in principle larger monitoring area. In the illustrated arrangement, itcan be determined, for example, by means of the acoustic pulses 17whether objectives 4 and/or further microscope components, e.g. animmersion device, are arranged on the objective revolver 3. The field ofview of the overview camera 9, on the other hand, is essentially limitedto the mirror 9B and does not cover other components on the objectiverevolver 3. Depending on the precision of the implemented sonar method,it is possible to distinguish between different models of immersiondevices and objectives.

FIG. 6

FIG. 6 shows schematically a process sequence of a further exampleembodiment of a method of the invention. In this case, the computingdevice of the microscopy system 100 is configured to identify ambientsounds from captured sounds.

An ambient activity 29, a closing of a door here, is shown by way ofexample as process P1″. Ambient sounds 28 (a slamming of the closingdoor) occur as a result.

In process P2, the at least one microphone captures sounds 22 containingthe ambient sounds 28.

In process P3, the captured sounds 22 are evaluated, e.g. by means of amachine-learned model M. The model M is trained to classify enteredsounds 22. An output 30 generated in process P4 by the model M is thusan identification 31 or an indication of a classification of the ambientactivity that caused the ambient sounds 28. In this example, the output30 or identification 31 thus indicates a closing of a room door.

In process P5, an action 34 occurs based on the identified ambientactivity, i.e. based on the output 30 of the model M. In the presentexample, the action 34 is a logging 37 of the identified ambientactivity 29. In particular, it is logged which measurement data werecaptured concurrently with the ambient activity 29 and may have beeninfluenced by the ambient activity 29. This makes it easier for a userto check for potentially compromised measurements during a measurementseries. In the event of an identification of potentially compromisedmeasurements, the ambient activity 29 can be identified as a source ofinterference easier and more reliably.

The computing device can also be designed to identify other ambientsounds such as those listed in the foregoing description. The exampleembodiment of FIG. 6 can also be combined with the example embodiment ofFIG. 2 or FIG. 4 . Captured sounds 22 can thereby be evaluated for bothany microscope sounds 25, 25′ and any ambient sounds 28 they contain.

The model M of FIG. 6 can be designed as described with reference toFIG. 3 wherein, however, the training data further comprises soundscontaining examples of ambient sounds that the model M is intended to beable to identify. The training data can accordingly include ambientsounds of, e.g., a slamming door, an opening of an incubation panel, apneumatic adjustment of a table supporting the microscope orconstruction noise, with a corresponding annotation. The microscopysystem 100 used in the embodiment of FIG. 6 can comprise the featuresdescribed in relation to the microscopy systems of FIG. 1 and/or FIG. 5.

The described example embodiments are purely illustrative and variantsof the same are possible within the scope of the attached claims.

LIST OF REFERENCE SIGNS

-   1 Microscope-   2 Stand-   3 Objective revolver-   4 (Microscope) objective-   5 Microphone-   5A Microphone array-   6 Sample stage-   7 Sample carrier-   8 Microscope camera-   9 Overview camera-   9A Field of view of the overview camera-   9B Mirror-   11 Immersion device-   12 Immersion fluid tank-   13 Immersion fluid line-   15 Sound transmitter-   16 Illumination device-   17 Acoustic pulses of the sound transmitter 15-   20 Height adjustment of the sample stage 6 causing a collision-   22, 22′ Sounds-   25, 25′ Microscope sounds-   26 Spectrogram-   27 Processed sounds-   28 Ambient sound-   29 Ambient activity, in particular closing of a door-   30 Output of the learned model M-   30 Identification or indication of a classification of a microscope    activity in progress-   34 Action/intervention in a microscope activity-   35 Stop command-   36 Instruction: Check immersion medium tank/immersion medium line-   37 Logging of the ambient activity-   70 Computing device-   80 Computer program-   100 Microscopy system-   A Annotations of the training data T1-T5-   M, M′ Learned model for evaluating captured sounds-   P1 Microscope activity/process: component collision-   P1′ Microscope activity/process: incorrect operation of immersion    device 11-   P1″, P2-P4, P2 b, P2, P4) Processes of methods according to the    invention-   T1-T5 Training data for learning the model M

What is claimed is:
 1. A microscopy system comprising: a microscope foranalyzing a sample; a computing device for processing measurementsignals; and at least one microphone for capturing sounds; wherein thecomputing device is configured to evaluate captured sounds in order toidentify a microscope activity in progress or command an intervention inthe microscope activity in progress based on microscope sounds.
 2. Themicroscopy system according to claim 1, wherein the computing device isconfigured to infer a defect of a microscope component based on themicroscope sounds; wherein the microscope component is a sample stage,an objective revolver, a laser scanner, an immersion device, a screw-onor otherwise releasably attachable component.
 3. The microscopy systemaccording to claim 1, wherein the computing device is configured toinfer an operating state of the microscope based on the microscopesounds; wherein the computing device is configured to detect one or moreof the following as an operating state: a differential interferencecontrast mode, based on a sound of a differential interference contrast(DIC) slider clicking into place as the microscope sound; a mounting ofan objective, based on a sound of an objective being screwed into anobjective revolver as the microscope sound; a sample stage movement,based on a sound of an operation of a manual sample stage as themicroscope sound; a defective support of a microscope component, basedon microscope sounds characteristic of a loose support of the microscopecomponent; an incorrect condenser position, based on movement sounds ofa swivel arm of a condenser when no sound of a complete pivoting of theswivel arm into place is detected.
 4. The microscopy system according toclaim 1, wherein the computing device is configured to detect one ormore of the following as microscope sounds and corresponding identifiedmicroscope activities: collision sounds of a collision betweenmicroscope components; cleaning sounds of a slide cleaning activity;sounds of an application of an immersion medium, wherein it isdistinguished based on these sounds between a correct immersion activityand an incorrect immersion activity in which air bubbles get into theimmersion medium; insertion sounds of a sample carrier insertionactivity at a sample stage; movement sounds of a filter wheel withfilters being rotated in or out of a microscope light path.
 5. Themicroscopy system according to claim 1, wherein the computing device isconfigured to detect a grating of a collision between an objective and acover slip as microscope sounds and, in the event of the detection ofsuch a collision, to stop a component movement in order to prevent abreaking of the cover slip.
 6. The microscopy system according to claim1, wherein the computing device is configured to also use contextualinformation for the identification of a microscope activity in progress,wherein the contextual information stems from an analysis of capturedoverview images or measurement data of a motion sensor or is informationregarding an initiated workflow of the microscope, a microscopeconfiguration used, an employed sample carrier or a current microscopeuser.
 7. The microscopy system according to claim 1, wherein thecomputing device is configured to monitor manual steps of a workflow bychecking whether microscope sounds caused by a user on the microscopeare typical of a current step of a predetermined workflow and when thestep is completed.
 8. The microscopy system according to claim 1, themicroscopy system further comprising an overview camera, wherein thecomputing device is configured to capture an overview image with theoverview camera in the event of an identification of a microscopeactivity based on the microscope sounds and to evaluate the overviewimage in order to verify or refine the identification of the microscopeactivity.
 9. The microscopy system according to claim 1, wherein thecomputing device is configured to establish a movement or manipulationof a sample, sample carrier or sample stage of the microscope from thecaptured sounds and accordingly command a new calibration or calibrationcontrol.
 10. The microscopy system according to claim 1, wherein thecomputing device is configured to also identify ambient sounds inaddition to microscope sounds from captured sounds.
 11. The microscopysystem according to claim 10, wherein the computing device is configuredto log which measurements occurred concurrently with a detected ambientsound or microscope sound.
 12. The microscopy system according to claim10, wherein the computing device is configured to carry out anidentification of microscope sounds or ambient sounds in response to asituation-dependent activation signal and not continuously, wherein thecomputing device is configured to generate the activation signal in theevent of certain workflows of the microscope or in the event of certainstates deduced from a visual monitoring.
 13. The microscopy systemaccording to claim 10, comprising a plurality of microphones and whereinthe computing device is configured to carry out an identification of themicroscope activity or of an ambient activity causing an ambient soundby evaluating captured sounds of the plurality of microphones dependingon a location of a sound source.
 14. The microscopy system according toclaim 1, wherein the computing device is configured to evaluate capturedsounds using a machine-learned model learned using training data of atleast one of microscope sounds or ambient sounds.
 15. The microscopysystem according to claim 14, wherein the model is learned using asupervised learning process in which the training data comprise at leastone of different microscope sounds or different ambient sounds, whichare respectively annotated with an annotation of an associatedmicroscope activity, wherein the training data comprise one or more ofthe following: cleaning sounds of a slide cleaning activity; othercleaning sounds that do not belong to a slide cleaning activity; soundsof an immersion device in the event of a correct application of animmersion medium; sounds of an immersion device in the event of anincorrect application of an immersion medium; insertion sounds of asample carrier insertion activity at a sample stage; other soundsproduced with the sample carrier when placed on a substrate without aninsertion activity being performed; sounds of a shock or blow to themicroscopy system; other sounds of a shock or blow which does notdirectly affect the microscopy system; collision sounds of microscopecomponents, including a grating of a collision between an objective anda cover slip, breaking sounds of a cover slip in the event of acollision with an objective, sounds of a collision between an objectiveor a condenser and different types of sample carriers, sounds of acollision between an objective and a sample stage; sounds of adifferential interference contrast (DIC) slider snapping into a DIC sloton the microscope; sounds of different filters snapping intocorresponding filter slots on the microscope; other snap-in soundsunrelated to the microscope; sounds of an objective being screwed intoan objective revolver; other sounds produced by a threaded attachmentunrelated to the microscope; movement sounds of a manual or motorizedsample stage in operation; movement sounds of a microscope componentwhich is movable by a motor or actuator; a rattling of an incorrectlyattached microscope component; operating sounds of a microscopecomponent in the event of a correct operation and in the event of anincorrect operation, wherein the microscope component is a sample stage,an objective revolver, an objective, an immersion device, a laserscanner or a screw-on or otherwise releasably attachable component;draft sounds, construction site noise, drilling sounds orair-conditioning sounds; sounds of a door closing or slamming next tothe microscopy system; sounds of footsteps or a person stumbling;hissing sounds of a pneumatic adjustment of a table on which themicroscope is supported; manual operating activity on ambient devices;photography sounds of a camera.
 16. The microscopy system according toclaim 14, wherein the model is learned using an unsupervised learningprocess in which the training data comprise at least one of differentmicroscope sounds and different ambient sounds captured during anerror-free operation of the microscope.
 17. The microscopy systemaccording to claim 1, further comprising a sound transmitter, whereinthe computing device is configured to control the sound transmitter toemit acoustic pulses, wherein the microphone measures reflected acousticpulses, and wherein the computing device is configured to establish apresence or location of objects by evaluating the acoustic pulsesmeasured by the microphone.
 18. The microscopy system according to claim1, wherein the computing device is configured to monitor workflows bychecking whether captured sounds are typical of a predeterminedworkflow.
 19. A microscopy system comprising: a microscope for analyzinga sample; a computing device for processing measurement signals; and atleast one microphone for capturing sounds; wherein the computing deviceis configured to evaluate captured sounds in order to identify ambientsounds.
 20. The microscopy system according to claim 19, wherein thecomputing device is configured to identify ambient sounds characteristicof a potentially disruptive external influence or ambient soundsrelating to one or more of the following: a shock or blow to themicroscopy system; a draft; a closing or slamming of a room door;footsteps or a stumbling of a person; a pneumatic adjustment of a tableon which the microscope is supported; construction site noise; drillingsounds; air-conditioning sounds; manual operating activity on ambientdevices; photography sounds of a camera, an opening or a closing of anincubator panel or door, an opening or a closing of a housing door. 21.A method for monitoring microscope activity, comprising: operating amicroscope; capturing sounds using at least one microphone; andevaluating the captured sounds in order to identify a microscopeactivity in progress or command an intervention in the microscopeactivity in progress based on microscope sounds.
 22. A computer programwith commands that, when executed by a computer, causes the execution ofthe method of claim 21.